Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Add Threat Match rule specific editable fields #200308

Merged

Conversation

maximpn
Copy link
Contributor

@maximpn maximpn commented Nov 15, 2024

Partially addresses: #171520

Summary

This PR adds is built on top of #193828 and #196948 and adds the following editable components for Threat Match rule type

  • threat_index
  • threat_query
  • threat_mapping
  • threat_indicator_path
  • threat_language threat_language was merged with threat_query

Details

This PR make a set of changes to make existing Threat Match form fields easily reusable as editable components and type safe when used in forms. In particular the following was done

  • Fixes a bug blocking Threat Match rules upgrading
  • Existing functionality was refactored to have reusable self-contained editable components for threat_index, threat_query, threat_mapping and threat_indicator_path rule fields
  • threat_language was removed since query type is included in threat_query field and can be edited with Query Bar
  • threat mapping input was split into separate component for individual fields to be reused
  • ThreatMatchComponent was refactored to be a controlled component instead of uncontrolled
    ThreatMatchComponent has a feature preventing users removing the single last entry. Instead deleting the last entry the delete button clears inputs. That functionality didn't work properly in Prebuilt Rule Customization workflow and rule creation/editing forms after creating a reusable ThreatMappingEdit component. Instead of trying to find a tricky fix ThreatMatchComponent was refactored to remove internal state. The feature preventing users removing the single last entry was reimplemented in ThreatMappingEdit component.
  • Fixes a bug reproducible in main where validation errors duplicated described in a comment
  • Fixes a bug reproducible in main allowing to save unknown source indices or indicator indices fields described in a comment

How to test

  • Ensure the prebuiltRulesCustomizationEnabled feature flag is enabled
  • Allow internal APIs via adding server.restrictInternalApis: false to kibana.dev.yaml
  • Clear Elasticsearch data
  • Run Elasticsearch and Kibana locally (do not open Kibana in a web browser)
  • Install an outdated version of the security_detection_engine Fleet package
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
  • Install prebuilt rules
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
  • Open a threat_match rule for editing. For example Threat Intel Hash Indicator Match with rule_id aab184d3-72b3-4639-b242-6597c99d8bca.

  • Edit Indicator index patterns, Indicator index query and/or Indicator filters, Indicator mapping and Indicator prefix override fields

  • Open Detection Rules (SIEM) Page -> Rule Updates -> click on Threat Intel Hash Indicator Match rule -> expand each Threat Match rule type specific field -> press Edit button

Screenshots

Threat Match Query edit component
image

Threat Match Index edit component
image

Threat Match Mapping edit component
image

Threat Match Indicator Path edit component
image

Threat Match Mapping unknown field names validation warnings
Screenshot 2024-12-18 at 12 45 41

Screenshot 2024-12-18 at 12 45 53 Screenshot 2024-12-18 at 12 47 05 Screenshot 2024-12-18 at 12 47 15

@maximpn maximpn added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area backport:version Backport to applied version labels v8.17.0 labels Nov 15, 2024
@maximpn maximpn self-assigned this Nov 15, 2024
@maximpn maximpn force-pushed the add-threat-match-specific-editable-fields branch 2 times, most recently from 018a0c9 to 9b576a1 Compare November 16, 2024 11:02
@maximpn maximpn requested a review from nikitaindik November 17, 2024 22:03
@maximpn maximpn marked this pull request as ready for review November 17, 2024 22:03
@maximpn maximpn requested review from a team as code owners November 17, 2024 22:03
@maximpn maximpn requested a review from vitaliidm November 17, 2024 22:03
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

Copy link
Contributor

@vitaliidm vitaliidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found few issues and left some comments, mostly questions and suggestions.
Will continue testing

@xcrzx
Copy link
Contributor

xcrzx commented Nov 20, 2024

I don’t have any changes in the filters field, but the diff shows some empty meta.alias fields that it thinks should be removed:

image

What is the meta field, and should it even be part of the diff calculation?

cc @dplumlee This seems similar to the issue we recently had with the schedule and threat fields.

@xcrzx
Copy link
Contributor

xcrzx commented Nov 20, 2024

The incorrect final field version has been selected. I have modifications to the field, so in this case, we should pre-select the current version and preserve all user modifications instead of removing them.

image

@nikitaindik nikitaindik removed their request for review November 21, 2024 10:51
@maximpn maximpn added v8.18.0 and removed v8.17.0 labels Nov 22, 2024
@maximpn maximpn force-pushed the add-threat-match-specific-editable-fields branch from e0ff1dd to f9e8e69 Compare November 27, 2024 21:50
@maximpn maximpn requested a review from vitaliidm November 27, 2024 21:52
@maximpn maximpn force-pushed the add-threat-match-specific-editable-fields branch from 52808bc to db006e0 Compare January 6, 2025 19:29
Copy link
Contributor

@nikitaindik nikitaindik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done, @maximpn! 👍 I've tested the PR locally and can confirm that Threat Match fields seem to be working well and rule upgrade works.

During testing found this header height issue, which was already fixed.
Left: this branch. Right: main.
Scherm­afbeelding 2025-01-06 om 19 58 31

Also in a separate PR we may consider adapting the Indicator Mapping field UI for a narrower container size. Right now it feels a little cramped.
Scherm­afbeelding 2025-01-06 om 20 57 15

@maximpn maximpn enabled auto-merge (squash) January 6, 2025 20:09
@maximpn maximpn merged commit 40f6628 into elastic:main Jan 7, 2025
8 checks passed
@maximpn maximpn deleted the add-threat-match-specific-editable-fields branch January 7, 2025 08:52
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12648207135

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 6488 6522 +34

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 22.1MB 22.1MB +2.6KB
Unknown metric groups

ESLint disabled line counts

id before after diff
securitySolution 574 573 -1

Total ESLint disabled count

id before after diff
securitySolution 657 656 -1

History

cc @maximpn

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [AI Connector] Migrates AI inference Connector to use a shared components from '@kbn/inference-endpoint-ui-common' (#204885)

Manual backport

To create the backport manually run:

node scripts/backport --pr 200308

Questions ?

Please refer to the Backport tool documentation

@maximpn
Copy link
Contributor Author

maximpn commented Jan 7, 2025

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

maximpn added a commit to maximpn/kibana that referenced this pull request Jan 7, 2025
…lastic#200308)

**Partially addresses:** elastic#171520

## Summary

This PR adds is built on top of elastic#193828 and elastic#196948 and adds the following editable components for Threat Match rule type

- threat_index
- threat_query
- threat_mapping
- threat_indicator_path
- ~~threat_language~~ `threat_language` was merged with `threat_query`

## Details

This PR make a set of changes to make existing Threat Match form fields easily reusable as editable components and type safe when used in forms. In particular the following was done

- Fixes a bug blocking Threat Match rules upgrading
- Existing functionality was refactored to have reusable self-contained editable components for `threat_index`, `threat_query`, `threat_mapping` and `threat_indicator_path` rule fields
- `threat_language` was removed since query type is included in `threat_query` field and can be edited with Query Bar
- threat mapping input was split into separate component for individual fields to be reused
- `ThreatMatchComponent` was refactored to be a controlled component instead of uncontrolled
 `ThreatMatchComponent` has a feature preventing users removing the single last entry. Instead deleting the last entry the delete button clears inputs. That functionality didn't work properly in Prebuilt Rule Customization workflow and rule creation/editing forms after creating a reusable `ThreatMappingEdit` component. Instead of trying to find a tricky fix `ThreatMatchComponent`  was refactored to remove internal state. The feature preventing users removing the single last entry was reimplemented in `ThreatMappingEdit` component.
- Fixes a bug reproducible in `main` where validation errors duplicated described in a [comment](elastic#200308 (comment))
- Fixes a bug reproducible in `main` allowing to save unknown source indices or indicator indices fields described in a [comment](elastic#200308 (comment))

## How to test

- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web browser)
- Install an outdated version of the `security_detection_engine` Fleet package
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```

- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```

- Open a `threat_match` rule for editing. For example `Threat Intel Hash Indicator Match` with rule_id `aab184d3-72b3-4639-b242-6597c99d8bca`.

- Edit `Indicator index patterns`, `Indicator index query` and/or `Indicator filters`, `Indicator mapping` and `Indicator prefix override` fields

- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Threat Intel Hash Indicator Match` rule -> expand each Threat Match rule type specific field -> press `Edit` button

## Screenshots

Threat Match Query edit component
<img width="1720" alt="image" src="https://github.com/user-attachments/assets/c7183ddf-8795-424c-90e4-b7eff14d9f69">

Threat Match Index edit component
<img width="1727" alt="image" src="https://github.com/user-attachments/assets/5e50cc98-6cc6-464d-a29d-89d31718482d">

Threat Match Mapping edit component
<img width="1725" alt="image" src="https://github.com/user-attachments/assets/aba6a723-0283-4b9e-80d2-376b1dea102e">

Threat Match Indicator Path edit component
<img width="1725" alt="image" src="https://github.com/user-attachments/assets/59aa12d9-377c-4c24-ab40-fef19e55e44e">

Threat Match Mapping unknown field names validation warnings
<img width="979" alt="Screenshot 2024-12-18 at 12 45 41" src="https://github.com/user-attachments/assets/0cfd8ae3-4865-49f8-a4ac-bafe19e01671" />

<img width="1094" alt="Screenshot 2024-12-18 at 12 45 53" src="https://github.com/user-attachments/assets/7f204e12-fe65-4a64-a029-1bb44ea366a3" />

<img width="2552" alt="Screenshot 2024-12-18 at 12 47 05" src="https://github.com/user-attachments/assets/53ac4612-f443-4d89-9474-8693ab9ced2d" />

<img width="2550" alt="Screenshot 2024-12-18 at 12 47 15" src="https://github.com/user-attachments/assets/1e345c88-9427-44ba-bc25-0164c39d1700" />

(cherry picked from commit 40f6628)

# Conflicts:
#	x-pack/platform/plugins/private/translations/translations/zh-CN.json
maximpn added a commit that referenced this pull request Jan 7, 2025
…lds (#200308) (#205681)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Add Threat Match rule specific editable fields
(#200308)](#200308)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-07T08:52:07Z","message":"[Security
Solution] Add Threat Match rule specific editable fields
(#200308)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR adds is built on top of
#193828 and
#196948 and adds the following
editable components for Threat Match rule type\r\n\r\n-
threat_index\r\n- threat_query\r\n- threat_mapping\r\n-
threat_indicator_path\r\n- ~~threat_language~~ `threat_language` was
merged with `threat_query`\r\n\r\n## Details\r\n\r\nThis PR make a set
of changes to make existing Threat Match form fields easily reusable as
editable components and type safe when used in forms. In particular the
following was done\r\n\r\n- Fixes a bug blocking Threat Match rules
upgrading\r\n- Existing functionality was refactored to have reusable
self-contained editable components for `threat_index`, `threat_query`,
`threat_mapping` and `threat_indicator_path` rule fields \r\n-
`threat_language` was removed since query type is included in
`threat_query` field and can be edited with Query Bar\r\n- threat
mapping input was split into separate component for individual fields to
be reused\r\n- `ThreatMatchComponent` was refactored to be a controlled
component instead of uncontrolled\r\n `ThreatMatchComponent` has a
feature preventing users removing the single last entry. Instead
deleting the last entry the delete button clears inputs. That
functionality didn't work properly in Prebuilt Rule Customization
workflow and rule creation/editing forms after creating a reusable
`ThreatMappingEdit` component. Instead of trying to find a tricky fix
`ThreatMatchComponent` was refactored to remove internal state. The
feature preventing users removing the single last entry was
reimplemented in `ThreatMappingEdit` component.\r\n- Fixes a bug
reproducible in `main` where validation errors duplicated described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869385209)\r\n-
Fixes a bug reproducible in `main` allowing to save unknown source
indices or indicator indices fields described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869412952)\r\n\r\n##
How to test\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled`
feature flag is enabled\r\n- Allow internal APIs via adding
`server.restrictInternalApis: false` to `kibana.dev.yaml`\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web browser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet package\r\n```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json' -H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d
'{\"force\":true}'
http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n```\r\n\r\n-
Install prebuilt rules\r\n```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123'
-H \"elastic-api-version: 1\" -d '{\"mode\":\"ALL_RULES\"}'
http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n```\r\n\r\n-
Open a `threat_match` rule for editing. For example `Threat Intel Hash
Indicator Match` with rule_id
`aab184d3-72b3-4639-b242-6597c99d8bca`.\r\n\r\n- Edit `Indicator index
patterns`, `Indicator index query` and/or `Indicator filters`,
`Indicator mapping` and `Indicator prefix override` fields\r\n\r\n- Open
`Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Threat
Intel Hash Indicator Match` rule -> expand each Threat Match rule type
specific field -> press `Edit` button\r\n\r\n##
Screenshots\r\n\r\nThreat Match Query edit component\r\n<img
width=\"1720\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/c7183ddf-8795-424c-90e4-b7eff14d9f69\">\r\n\r\nThreat
Match Index edit component\r\n<img width=\"1727\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/5e50cc98-6cc6-464d-a29d-89d31718482d\">\r\n\r\nThreat
Match Mapping edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/aba6a723-0283-4b9e-80d2-376b1dea102e\">\r\n\r\nThreat
Match Indicator Path edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/59aa12d9-377c-4c24-ab40-fef19e55e44e\">\r\n\r\nThreat
Match Mapping unknown field names validation warnings\r\n<img
width=\"979\" alt=\"Screenshot 2024-12-18 at 12 45 41\"
src=\"https://github.com/user-attachments/assets/0cfd8ae3-4865-49f8-a4ac-bafe19e01671\"
/>\r\n\r\n<img width=\"1094\" alt=\"Screenshot 2024-12-18 at 12 45 53\"
src=\"https://github.com/user-attachments/assets/7f204e12-fe65-4a64-a029-1bb44ea366a3\"
/>\r\n\r\n<img width=\"2552\" alt=\"Screenshot 2024-12-18 at 12 47 05\"
src=\"https://github.com/user-attachments/assets/53ac4612-f443-4d89-9474-8693ab9ced2d\"
/>\r\n\r\n<img width=\"2550\" alt=\"Screenshot 2024-12-18 at 12 47 15\"
src=\"https://github.com/user-attachments/assets/1e345c88-9427-44ba-bc25-0164c39d1700\"
/>","sha":"40f6628c220217fa5bebcc546d21730ccf754d90","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0"],"number":200308,"url":"https://github.com/elastic/kibana/pull/200308","mergeCommit":{"message":"[Security
Solution] Add Threat Match rule specific editable fields
(#200308)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR adds is built on top of
#193828 and
#196948 and adds the following
editable components for Threat Match rule type\r\n\r\n-
threat_index\r\n- threat_query\r\n- threat_mapping\r\n-
threat_indicator_path\r\n- ~~threat_language~~ `threat_language` was
merged with `threat_query`\r\n\r\n## Details\r\n\r\nThis PR make a set
of changes to make existing Threat Match form fields easily reusable as
editable components and type safe when used in forms. In particular the
following was done\r\n\r\n- Fixes a bug blocking Threat Match rules
upgrading\r\n- Existing functionality was refactored to have reusable
self-contained editable components for `threat_index`, `threat_query`,
`threat_mapping` and `threat_indicator_path` rule fields \r\n-
`threat_language` was removed since query type is included in
`threat_query` field and can be edited with Query Bar\r\n- threat
mapping input was split into separate component for individual fields to
be reused\r\n- `ThreatMatchComponent` was refactored to be a controlled
component instead of uncontrolled\r\n `ThreatMatchComponent` has a
feature preventing users removing the single last entry. Instead
deleting the last entry the delete button clears inputs. That
functionality didn't work properly in Prebuilt Rule Customization
workflow and rule creation/editing forms after creating a reusable
`ThreatMappingEdit` component. Instead of trying to find a tricky fix
`ThreatMatchComponent` was refactored to remove internal state. The
feature preventing users removing the single last entry was
reimplemented in `ThreatMappingEdit` component.\r\n- Fixes a bug
reproducible in `main` where validation errors duplicated described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869385209)\r\n-
Fixes a bug reproducible in `main` allowing to save unknown source
indices or indicator indices fields described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869412952)\r\n\r\n##
How to test\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled`
feature flag is enabled\r\n- Allow internal APIs via adding
`server.restrictInternalApis: false` to `kibana.dev.yaml`\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web browser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet package\r\n```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json' -H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d
'{\"force\":true}'
http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n```\r\n\r\n-
Install prebuilt rules\r\n```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123'
-H \"elastic-api-version: 1\" -d '{\"mode\":\"ALL_RULES\"}'
http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n```\r\n\r\n-
Open a `threat_match` rule for editing. For example `Threat Intel Hash
Indicator Match` with rule_id
`aab184d3-72b3-4639-b242-6597c99d8bca`.\r\n\r\n- Edit `Indicator index
patterns`, `Indicator index query` and/or `Indicator filters`,
`Indicator mapping` and `Indicator prefix override` fields\r\n\r\n- Open
`Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Threat
Intel Hash Indicator Match` rule -> expand each Threat Match rule type
specific field -> press `Edit` button\r\n\r\n##
Screenshots\r\n\r\nThreat Match Query edit component\r\n<img
width=\"1720\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/c7183ddf-8795-424c-90e4-b7eff14d9f69\">\r\n\r\nThreat
Match Index edit component\r\n<img width=\"1727\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/5e50cc98-6cc6-464d-a29d-89d31718482d\">\r\n\r\nThreat
Match Mapping edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/aba6a723-0283-4b9e-80d2-376b1dea102e\">\r\n\r\nThreat
Match Indicator Path edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/59aa12d9-377c-4c24-ab40-fef19e55e44e\">\r\n\r\nThreat
Match Mapping unknown field names validation warnings\r\n<img
width=\"979\" alt=\"Screenshot 2024-12-18 at 12 45 41\"
src=\"https://github.com/user-attachments/assets/0cfd8ae3-4865-49f8-a4ac-bafe19e01671\"
/>\r\n\r\n<img width=\"1094\" alt=\"Screenshot 2024-12-18 at 12 45 53\"
src=\"https://github.com/user-attachments/assets/7f204e12-fe65-4a64-a029-1bb44ea366a3\"
/>\r\n\r\n<img width=\"2552\" alt=\"Screenshot 2024-12-18 at 12 47 05\"
src=\"https://github.com/user-attachments/assets/53ac4612-f443-4d89-9474-8693ab9ced2d\"
/>\r\n\r\n<img width=\"2550\" alt=\"Screenshot 2024-12-18 at 12 47 15\"
src=\"https://github.com/user-attachments/assets/1e345c88-9427-44ba-bc25-0164c39d1700\"
/>","sha":"40f6628c220217fa5bebcc546d21730ccf754d90"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200308","number":200308,"mergeCommit":{"message":"[Security
Solution] Add Threat Match rule specific editable fields
(#200308)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR adds is built on top of
#193828 and
#196948 and adds the following
editable components for Threat Match rule type\r\n\r\n-
threat_index\r\n- threat_query\r\n- threat_mapping\r\n-
threat_indicator_path\r\n- ~~threat_language~~ `threat_language` was
merged with `threat_query`\r\n\r\n## Details\r\n\r\nThis PR make a set
of changes to make existing Threat Match form fields easily reusable as
editable components and type safe when used in forms. In particular the
following was done\r\n\r\n- Fixes a bug blocking Threat Match rules
upgrading\r\n- Existing functionality was refactored to have reusable
self-contained editable components for `threat_index`, `threat_query`,
`threat_mapping` and `threat_indicator_path` rule fields \r\n-
`threat_language` was removed since query type is included in
`threat_query` field and can be edited with Query Bar\r\n- threat
mapping input was split into separate component for individual fields to
be reused\r\n- `ThreatMatchComponent` was refactored to be a controlled
component instead of uncontrolled\r\n `ThreatMatchComponent` has a
feature preventing users removing the single last entry. Instead
deleting the last entry the delete button clears inputs. That
functionality didn't work properly in Prebuilt Rule Customization
workflow and rule creation/editing forms after creating a reusable
`ThreatMappingEdit` component. Instead of trying to find a tricky fix
`ThreatMatchComponent` was refactored to remove internal state. The
feature preventing users removing the single last entry was
reimplemented in `ThreatMappingEdit` component.\r\n- Fixes a bug
reproducible in `main` where validation errors duplicated described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869385209)\r\n-
Fixes a bug reproducible in `main` allowing to save unknown source
indices or indicator indices fields described in a
[comment](https://github.com/elastic/kibana/pull/200308#discussion_r1869412952)\r\n\r\n##
How to test\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled`
feature flag is enabled\r\n- Allow internal APIs via adding
`server.restrictInternalApis: false` to `kibana.dev.yaml`\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web browser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet package\r\n```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json' -H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d
'{\"force\":true}'
http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n```\r\n\r\n-
Install prebuilt rules\r\n```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123'
-H \"elastic-api-version: 1\" -d '{\"mode\":\"ALL_RULES\"}'
http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n```\r\n\r\n-
Open a `threat_match` rule for editing. For example `Threat Intel Hash
Indicator Match` with rule_id
`aab184d3-72b3-4639-b242-6597c99d8bca`.\r\n\r\n- Edit `Indicator index
patterns`, `Indicator index query` and/or `Indicator filters`,
`Indicator mapping` and `Indicator prefix override` fields\r\n\r\n- Open
`Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Threat
Intel Hash Indicator Match` rule -> expand each Threat Match rule type
specific field -> press `Edit` button\r\n\r\n##
Screenshots\r\n\r\nThreat Match Query edit component\r\n<img
width=\"1720\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/c7183ddf-8795-424c-90e4-b7eff14d9f69\">\r\n\r\nThreat
Match Index edit component\r\n<img width=\"1727\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/5e50cc98-6cc6-464d-a29d-89d31718482d\">\r\n\r\nThreat
Match Mapping edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/aba6a723-0283-4b9e-80d2-376b1dea102e\">\r\n\r\nThreat
Match Indicator Path edit component\r\n<img width=\"1725\" alt=\"image\"
src=\"https://github.com/user-attachments/assets/59aa12d9-377c-4c24-ab40-fef19e55e44e\">\r\n\r\nThreat
Match Mapping unknown field names validation warnings\r\n<img
width=\"979\" alt=\"Screenshot 2024-12-18 at 12 45 41\"
src=\"https://github.com/user-attachments/assets/0cfd8ae3-4865-49f8-a4ac-bafe19e01671\"
/>\r\n\r\n<img width=\"1094\" alt=\"Screenshot 2024-12-18 at 12 45 53\"
src=\"https://github.com/user-attachments/assets/7f204e12-fe65-4a64-a029-1bb44ea366a3\"
/>\r\n\r\n<img width=\"2552\" alt=\"Screenshot 2024-12-18 at 12 47 05\"
src=\"https://github.com/user-attachments/assets/53ac4612-f443-4d89-9474-8693ab9ced2d\"
/>\r\n\r\n<img width=\"2550\" alt=\"Screenshot 2024-12-18 at 12 47 15\"
src=\"https://github.com/user-attachments/assets/1e345c88-9427-44ba-bc25-0164c39d1700\"
/>","sha":"40f6628c220217fa5bebcc546d21730ccf754d90"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
@banderror
Copy link
Contributor

Also in a separate PR we may consider adapting the Indicator Mapping field UI for a narrower container size. Right now it feels a little cramped.

@nikitaindik @maximpn Can we please create a bug for that?

@maximpn
Copy link
Contributor Author

maximpn commented Jan 7, 2025

Can we please create a bug for that?

@banderror I created #205722.

kowalczyk-krzysztof pushed a commit to kowalczyk-krzysztof/kibana that referenced this pull request Jan 7, 2025
…lastic#200308)

**Partially addresses:** elastic#171520

## Summary

This PR adds is built on top of elastic#193828 and elastic#196948 and adds the following editable components for Threat Match rule type

- threat_index
- threat_query
- threat_mapping
- threat_indicator_path
- ~~threat_language~~ `threat_language` was merged with `threat_query`

## Details

This PR make a set of changes to make existing Threat Match form fields easily reusable as editable components and type safe when used in forms. In particular the following was done

- Fixes a bug blocking Threat Match rules upgrading
- Existing functionality was refactored to have reusable self-contained editable components for `threat_index`, `threat_query`, `threat_mapping` and `threat_indicator_path` rule fields 
- `threat_language` was removed since query type is included in `threat_query` field and can be edited with Query Bar
- threat mapping input was split into separate component for individual fields to be reused
- `ThreatMatchComponent` was refactored to be a controlled component instead of uncontrolled
 `ThreatMatchComponent` has a feature preventing users removing the single last entry. Instead deleting the last entry the delete button clears inputs. That functionality didn't work properly in Prebuilt Rule Customization workflow and rule creation/editing forms after creating a reusable `ThreatMappingEdit` component. Instead of trying to find a tricky fix `ThreatMatchComponent`  was refactored to remove internal state. The feature preventing users removing the single last entry was reimplemented in `ThreatMappingEdit` component.
- Fixes a bug reproducible in `main` where validation errors duplicated described in a [comment](elastic#200308 (comment))
- Fixes a bug reproducible in `main` allowing to save unknown source indices or indicator indices fields described in a [comment](elastic#200308 (comment))

## How to test

- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web browser)
- Install an outdated version of the `security_detection_engine` Fleet package
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```

- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```

- Open a `threat_match` rule for editing. For example `Threat Intel Hash Indicator Match` with rule_id `aab184d3-72b3-4639-b242-6597c99d8bca`.

- Edit `Indicator index patterns`, `Indicator index query` and/or `Indicator filters`, `Indicator mapping` and `Indicator prefix override` fields

- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Threat Intel Hash Indicator Match` rule -> expand each Threat Match rule type specific field -> press `Edit` button

## Screenshots

Threat Match Query edit component
<img width="1720" alt="image" src="https://github.com/user-attachments/assets/c7183ddf-8795-424c-90e4-b7eff14d9f69">

Threat Match Index edit component
<img width="1727" alt="image" src="https://github.com/user-attachments/assets/5e50cc98-6cc6-464d-a29d-89d31718482d">

Threat Match Mapping edit component
<img width="1725" alt="image" src="https://github.com/user-attachments/assets/aba6a723-0283-4b9e-80d2-376b1dea102e">

Threat Match Indicator Path edit component
<img width="1725" alt="image" src="https://github.com/user-attachments/assets/59aa12d9-377c-4c24-ab40-fef19e55e44e">

Threat Match Mapping unknown field names validation warnings
<img width="979" alt="Screenshot 2024-12-18 at 12 45 41" src="https://github.com/user-attachments/assets/0cfd8ae3-4865-49f8-a4ac-bafe19e01671" />

<img width="1094" alt="Screenshot 2024-12-18 at 12 45 53" src="https://github.com/user-attachments/assets/7f204e12-fe65-4a64-a029-1bb44ea366a3" />

<img width="2552" alt="Screenshot 2024-12-18 at 12 47 05" src="https://github.com/user-attachments/assets/53ac4612-f443-4d89-9474-8693ab9ced2d" />

<img width="2550" alt="Screenshot 2024-12-18 at 12 47 15" src="https://github.com/user-attachments/assets/1e345c88-9427-44ba-bc25-0164c39d1700" />
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:version Backport to applied version labels Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants